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ABSTRACT 

This study reports findings from a survey of U.S. and 
Canadian medical libraries belonging to the Association of Academic 
Health Sciences Library Directors (AAHSLD). Conducted in the spring 
of 1990 and fundjd by AAHSLD, the survey had the two primary purposes 
of assessing th*j use of the AAHSLD "Annual Statistics" in providing 
data for decision making, and offering recommendations to improve the 
usefulness of the "Annual Statistics.' Usable responses were received 
from 83 of the 144 medical libraries surveyed. Both quantitative and 
qualitative assessments were made on the survey data, and the 
findings generally suggest satisfaction with the "Annual Statistics." 
However, the "Annual Statistics" could be improved by including more 
performance measures and categorizing some of the data differently. 
Other findings and specific recommendations to improve the 
publication are included in the report. A copy of the questionnaire 
and comments on individual questions by the respondents are appended. 
(7 references) (Author/SD) 
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I. INTRODUCTION 



The survey, The Use of the AAHSLD Annual Statistics in 
Member Libraries, was developed and analyzed by Carol A. Hert and 
Charles R. McClure for the Statistics Comiaitte of the Association 
of Academic Health Sciences Library Directors (AAHLSD) . The 
survey had several objectives. These were to assess: 

- how AAHLSD members were currently using quantitative 
data for decision making, 

- the use of the AAHLSD Annual Statistics in providing 
data for decision making, 

- the current and potential use of performance measures 
for decision making, and 

- possible changes to the Annual Statistics so that its 
usefulness to members could be improved. 

The intent of this report is to present the results of the 
survey along with additional questions which were raised during 
analysis of the data. While our goal in tnis report is not to 
formally review the body of literature which exists on decision 
making and the use of quantitative data, it is important to set 
the survey and its results in a larger context. 

With increasing budgetary pressures on libraries, and 
patrons who are ever more sophisticated and demanding, library 
administrators have become increasingly concerned with managing 
their decision making processes better so that resources can be 
allocated more effectively and efficiently. McClure, paraphras- 
ing Eden and Harris, defines decision making as "that process 
whereby information is converted into action [and] is largely 
concerned with the process of acquiring, controlling, and uti- 
lizing information to accomplish some objective" (McClure, 1986, 
p. 324) . Managers are turning to the application of quantitative 
data to aid in this decision making (ART SPEC Kit no. 153, 1989; 
Cronin, 1985; and McClure, 1984; 1986). 

Traditionally, the generation of statistics from "similar" 
library types has been used by library managers for a variety of 
decision making purposes. Unfortunately, these data are often of 
limited utility for two reasons. They tend to report inputs 
and/or use of resources rather than outputs (services; or the 
quality or impact of these service5=, and the da*" a are often 
viewed in isolation from the mission, goals, user needs, or 
organizational constraints of the reporting libraries. In recog- 
nition .of these concerns, there has been an increasing awareness 
of the value of performance measures, that is^r data which measure 
the effectiveness and efficiency of library services and ac- 
tivities (Cronin, 1985; Van House, et. al., 1987.; Van House, 
Weil, and McClure, 1990). Indeed, the Association of College and 
Research Libraries contracted for the development of a perfor- 
mance measures manual for academic and research libraries which 
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will be published in June oi 1990 (Van House, Weil, and McClure, 
1990). Performance measure data enable a manager to assess the 
quality of services and activities, which may be more useful to 
planning than the analysis of data which merely report the nature 
of current operations (McClure, 1984). 

What still seems to be less than clear is the connection 
between statistics and performance measures and their use in 
decision making (ARL SPEC Kit no. 153, 1989). The results of the 
current survey are valuable in that they not only provide data 
for meeting the objectives outlined above, but also address the 
key research question of how quantitative data, in general, and 
performance measures, in particular, can be better collected, 
tabulated, and presented to improve decision making in all types 
of libraries. Some of the findings from this survey suggest 
possible strategies to address this question. 



II • METHODOLOGY 



QUESTIONNAIRE DEVELOPMENT 

Usage of the AAHSLD Annual Statistics by member libraries 
we.- assv.ssed through a self -administered mail survey (Appendix 
A). The survey was prepared anJl analyzed by Carol A. Hert and 
Charles R. McClure. The questionnaire was developed based on the 
study objectives enumerated above. The draft questionnaire was 
reviewed by members of the Statistics Committee and a pretest 
done by a director of an AAHLSLD member library. Following the 
initial review and subsequent revisions, a second draft was 
reviewed by a second medical library director and his admini- 
strative staff and minor revisions made. Tho finalized question- 
naire was mailed with a self addressed stamped envelope to all 
members of the AAHSLD in March, 1990 with a requested return date 
of April 15, 1990. 

RETURN RATE 

83 surveys were returned by April 25, 1990 and were anal- 
yzed. An additional 13 (total of 96) were returned after April 
25; too late to be included in the quantitative analysis. Open- 
ended responses from all 96 questionnaires were recorded. 

PROPORTIONAL REPRESENTATION 

The 83 surveys returned by April 25, 1990 were analyzed to 
determine whether they were representative of the total popula- 
tion of AAHSLD libraries. Proportional representation was impor- 
tant in assuring that results of further analyses would indicate 
the feelings of AAHSLD ^nember libraries* as a whole. Return rate 
for each group of libraries (as determined from the 1987-1988 
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Annual Statistics , pp. 174-196) was compared to the number of 
libraries in each group. Using postmark information, we were able 
to determine the correct category for 75 of the 83 surveys 
returned. 8 surveys were un-postmarked and were not included in 
the analysis. 

The numbers and proportions are presented in Table 1 - 
Response Rate by Library Group. A Chi square test of indepen- 
dence was performed to test whether the proportion of surveys 
returned from each group was significantly different from the 
proportion of libraries in that group in the population of AAHLSD 
libraries. 



TABLE 1 

RESPONSE RATE BY LIBRARY GROUP 



Group 


Number of 


Number of 


% of 


% of 




Libraries 


Returned 


Libraries 


Seturne 






Surveys 






I 


14 


8 


9.7 


10.7 


II 


33 


13 


22.9 


17.3 


III 


50 


24 


34.7 


32.0 


IV 


17 


11 


11.8 


14.7 


non-cate- 










gorized* 


30 


19 


20.8 


25.3 


total 


144 


75 


99.9** 


100.0 


* AAHSLD 


libraries not 


included in 


the categories 


** round' 


ng error 









The Chi square test indicated no significant difference 
between the group proportions in the returned surveys and in the 
population (test statistic = 1.64, region of rejection for Chi 
square with 4 degrees of freedom > 9.49), and we continued with 
the analyses assured that the surveys returned were represent- 
ative of the population of AAHSLD libraries. 

QUANTITATIVE 3^ALYSES 

For all closed-end questions, totals and proportions were 
calculated. Where appropriate, averages were also calculated. 
In addition to these descriptive statistics, a correlation 
coefficient was calculated between questions 1 and 6 and a number 
of Chi Square tests of independence were calculated. These tests 
will be described more thoroughly in the Results section. SAS 
statistical software was used for the quantitative analyses (SAS 
Institute Inc., 1985). 
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QDAliITATIVB l^ALYSES 



All open-ended responses from the 96 questionnaires Wxiich 
were returned by May 10, 1390 were transcribed (Appendix B - 
Comments) . The comments were then analyzed and major trends 
identified. 

LIMITATIONS 

While the survey was able to provide a variety of useful 
information concerning the use of the Annual Statistics volume, 
it, like all other self -reporting surveys, may not be a complete- 
ly accurate picture of the use. People frequently have problems 
reporting their activities in such a format. We have taken the 
inherent inadequacy of self -reporting surveys into account in our 
analysis, and will suggest some alternative research approaches 
which might be used to validate the data collected with this 
suirvey . 



III. RESULTS 



This section includes both the quantitative and qualitative 
analyses. Results are organized into 6 sections: 

- AAHSLD Members* Use of Quantitative Data 

- Performance Measures 

- Annual Statistics Data: Amount, Types, Validity, Reliabil- 

ity 

- Annual Statistics Data Preparation and Products 

- The Categorization Scheme 

- Summarization of Trends in Open-ended Comments 

Generally, the results suggest satisfaction with the Annual 
Statistics , but a number of key issues to be addressed were also 
identified. 



AAHSLD MEMBERS' USE OP QUANTITATIVE DATA 

Question 1 on the survey assessed the uses AAHSLD members 
make of quantitative data for decision making in their libraries. 
Table ^ - Uses of Quantitative Data indicates the number and 
proportion of respondents who checked each use in question 1. 



TABL2 2 
USES OP QUANTITATIVE DATA 



Use made of quantitative data 



Number of 
respondents 



Providing comparisons between 

your library and others 
Providing supporting data for 

budget requests 
Providing salary information 
Establishing benchmarks to track 

library development over time 
Providing data for intra-library reports 
Providing data for library resources 

allocation process 
Establishing benchmarks for 

productivity standards 
Providing early warning signals of 

trouble spots in library operation 
Calculating performance measures 
Providing tools to calculate 

management formulas 
Other 



% of total 
respondents 
(N=83) 



81 


97.6 


•77 


92.8 


65 


78.3 


61 


73.5 


40 


48.2 


33 


39.8 


31 


37.3 


29 


34.9 


25 


30.1 


23 


27.7 


5 


6.0 



Average number of uses checked by a respondent = 5.7 



Question 6 asked respondents to indicate the importance of 
the Annual Statistics in providing data for the same uses repre- 
sented in question 1. Respondents were asked to indicate whether 
the Annual Statistics was essential (value=l) , somewhat important 
(value=^2) , or not important (value==3) in providing data for each 
activity. The average value for each activity and the number of 
respondents is indicated in Table 3 - Importance of the Annual 
Statistics. The lower the average, the mere important rei=!pon- 
dents, as a group, considered the Annual Statistics for providing 
data. 
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TABLE 3 

IMPORTANCE OP THE ANNUAL STATISTICS 



Use 



Average Number of 
value* respondents** 



Providing comparisons between your 

library and others 
Providing supporting data for 

budget requests 
Providing salary information 
Establishing benchmarks to 

track library development 
Establishing benchmarks for 

productivity standards 
Providing data for library resources 

allocation process 

Other 

Calculating performance measures 
Providing data for intra-library 
reports 

Providing early warning signals 

of trouble spots in operations 
Providing tools for management 

formulas 

* values were Inessential, 2-somewhat important, 3=not important 
Indicates the number of respondents out of 83 respondents who 
rieported values of l, 2, or 3 



1.19 


80 


1.48 


82 


± . 0 o 


79 


1.63 


79 


1.96 


72 


2 . 00 


73 


2 . 00 


5 


2.06 


66 


2.17 


77 


2.25 


73 


2.29 


68 



In order to determine if respondents' use of the Annual 
Statistics correlated to the ways they use data for decision 
making, a comparison between the rankings of question l and 
question 6 was calculated ::ising the Spearman Rho correlation 
coefficient. The number of respondents who responded yes for 
each use indicated in question i was correlated with the total of 
respondents who indicated an importance of l or 2 for each use on 
question 6. A Spearman Rho of .955 was found (significant at the 
.05 alpha level, with 9 degrees of freedom) indicating a substan- 
tial correla\:ion between the rankings in question i and 6 
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The three major implications of this correlation are: 



- There are two distinct clusters of uses made of quan- 
titative data (the first cluster includes uses for 
external comparisons and justifying budget requests and 
the second includes uses for internal comparisons.) 

- The Annual Statistics is considered very important in 
providing data for the most frequent uses of quant- 
itative data indicated in question 1. 

- There is a strong correlation between the rankings of 
uses for quantitative data in general and the rankings 
of the uses for which the Annual Statistics is useful 
(either essential or somewhat important) in providing 
data. 

Examination of Table 2 reveals that there are four uses 
commonly made of quantitative data for decision making. These 
are for 

- providing comparisons between the library and others, 

- providing supporting data for budget requests, 

- providing salary information, and 

- establishing benchmarks to track library development 
over time. 

These four most highly cited uses of quantitative data had 
response percentages ranging from 73.5 to 97.6. The response 
percentage is the number of respondents who checked that use 
divided by the number of respondents (83) . These four uses are 
clearly separated from the other uses which show much lower but 
also fairly consistent response percentages from 27.7 to 48.2. 

When responses to question 6 are examined, a similar pattern 
is revealed. The four uses for which the Annual Statistics was 
considered the most important for providing data (with averages 
ranging from l.?,9 to 1.63) are the same four uses reported most 
frequently by respondents in question 1 as uses of quantitative 
data. Table 3 also shows that there is no use for which all 
respondents considered the Annual Statistics to be not useful. 
(Such ^an activity would have had an average value of 3.0.) 

The findings suggest that library managers use quantitative 
data for determining their positions relative to others and for 
convincing administrations to provide more support to the lib- 
rary. Additionally, we suggest that the high correlation coeffi- 
cient indicates that tha Annual Statistics is a valuable source 
of quantitative data for AAHSLD members that is particularly 
effective for providing information to support respondents* most 
frequent uses of quantitative data. Additional research would be 
necessary to determine whether the Annual Statistics is used 
exclusively or in conjuntion with other sources of data. 
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FERFORNGaNCE HEABURES 



Questions 2 through 5 on the survey assessed members' 
current and potential use of performance measures to aid in 
decision making, with performance measures being defined as data 
which measure the effectiveness and/or quality of library ac- 
tivities and services. When asked if they currently made use of 
performance measures, 

- 31 (39.3%) responded that they currently used perfor- 
mance measures. 

- 48 (62 8%) responded that they were not using them. 

Examination of the 39 responses to the inquiry of what 
performance measures were in use indicated that many respondents 
dM not have a good understanding of a performance measure. Some 
people mentioned methods and sources for obtaining data (e.g. RML 
reports, rime/motion stud-*es, user studies) rather than perfor- 
mance measures. Specific performance measures mentioned as being 
in use were of two types: 

- measures of service to the user (e.g. collection 
development funds per user, ILL requests to ILL re- 
quests filled) . 

measures of efficiency of internal operations (ILL 
activity to ILL staffing) . 

When asked in question 3 whether more performance measure 
data should be included in the Annual Statistics , 

- 38 (45.8%) responded yes. 

- 36 (43.4%) responded no^ 

- 9 (10.8%) gave no response. 

39 respondents provided examples of potential performance 
measure data to be included. As with the open-endedL responses to 
question 2, many of the examples could not be considered perfor- 
mance measures by our definition. 

While respondents were equally divided on whether more 
performance data should be incluaed in the Annual Statistics ^ 
when asked in question 5 whether they would be willing to report 
more data in order to allow presentation of more performance 
measures in the Annual St at istics , 

- 62 (75.7%) said they would be willin'-; to report more 
data . 

- 17 (20.5?.) said no, they would not be willing. 

- 4 (4.8%) said it depends and that libraries would need 
advance notice of what new data would be needed. 
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The difference between the results of question 3 and ques- 
tion 5 gives rise to several speculations. Open-ended responses 
to questions 2 and 4 suggest that respondents are confused about 
the nature of performance measures. Further investigation is 
needed to dete-rxaine if confusion, or some other factor, is the 
cause of the different responses reported in questions 3 and 5. 

Responses to question 5 suggest that rejspcndents do, in 
fact, want more performance measures reported in the Annual 
Statistics . However, that might not hold true if the amount of 
time and effort necessary to collect and report performance 
measures is too great. It may be possible to recombine already 
collected data to present additional performance measures. 

We examined the ^-^.sponses to qizestion 2 and 4 as veil as t^*-^ 
Annual Statistics volume to see whether data could be recombintv^ 
to provide additional performance measures without new data 
collection. Clientele figures are currently reported and could 
iDe combined with data concerning collection size, expenditures, 
facility size, or ILL activity to generate performance data 
measuring service to users. Some examples of possible measures 
are: 

- expenditures per user 

- seating a'^^ailability per user 

- ILL requests per user (as a measure of collection 
strength) 

operating costs per user 

- serial subscriptions per user 

Performance measures designed to measure the ef fe<.tiveness of 
internal library operations coi.ld also be generated. Lil:raries 
are currently reporting staffing levels and expenditures and 
these data could be combined with collection data to produce 
additional performance measures such as: 

- cataloging (or other unit) output per FTE. 
expenditures per unit output. 

Members of the AAHSLD may ^ish to review the existing data 
elements to determine 

- what types of performance measures could be calculated 
from these data elements, and 

- the degree to which these performance measures could 
provide useful management information to member librar- 
ies. 
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ANNUAL STATISTICS DATA: AMOUNT, TYPES, VALIDITY, RELIABILITY 

Several questions in the survey assessed respondents' 
perceptions of the amourt of data collected, the value of that 
data, and the data's reliability and validity. In response to 
cmestion 7, regarding the amount of data in the Annual Statis- 
tics , 

- 57 (78.1%) felt the right amount of data was being 
collected. 

- 11 (13.3%) felt too much was being collected. 

- 8 (9.6%) felt too little was being collected. 

- 7 (8.4%) did not respond. 

Respondents were asked in question 8 which data were espe- 
cially useful. 68 (81.9%) out of 83 respondents commented. 
Almost every type of table in the Annual St atistics was men- 
tioned, but those mentioned most frequently were the tables 
depicting 

- salaries, (36 respondents). 

- expenditures across all library functions, (30 respon- 
dents) . 

- collection data including collection size and current 
acquisitions figures, (23 respondents). 

If respondents commented on how these data ^'ere used, they 
cited the need to convince administrators in budget requests, and 
to determine their library's ranking among comparative libraries. 

44 (53.0%) respondents provided comments in question 9'^ 
concerning which data were not useful. Again, as with question 8, 
there was a large amoum f variation in the responses. Two 
concerns stood out. 

- Respondents (18) commented on the potential lack of 
uniformity in some counts (e.^. reference transactions, 
clientele counts) . 

- 7 respondents specifically cited the photocopy exposure 
count as not being useful. 

Questions 14 and 15 assessed respondents* perceptions of the 
reliability and validity of the data, respectively. While open- 
ended responses to question 9 showed some concern for reliability 
and validity issues, when asked to indicate perceptions using a 
Likert scale, averages fell in the moderately accurate range. 
Table 4 - Reliability and Validi+-y Scores indicates the average 
values across all respondents for leliability and validity. 
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TABLE 4 

RELIABILITY AND VALIDITY SCORES 



Data Category kverage Average 

Reliability* Validity* 

(N=65)** (N=78)** 

Clientele 3.00 3.25 

Collections 3.50 3.49 

Expenditures 3.60 3. 65 

Facilities 3.97 3.73 

Seirvices 3.05 3! 13 

Staffing and 

Salaries 3.98 3.91 

* On a five point scale with l=not accurate and 5= 
extremely accurate 

** Not all respondents answered these questions 



While It may be somewhat reassuring that respondents believe 
the data are "moderately accurate," additional on-site investiga- 
tion m a sample of libraries would be needed to determine the 
actual quality of data being reported. 

ANNUAL S TATISTICS DATA PREPARATION AND PRODUCTS 

Another series o"f questions focussed on the time spent by 
member libraries in the preparation of the data to be submitted 
and on the results of that labor. Question 10 asked respondents 
to report the amount of time they spent preparina the data and 
their feelings about the amount of time spent. 

- 17.85 hours was the average amount of time spent 
preparing the data. 

- 50 (60.2%) respondents thought that the amount of time 
they spent was about right. 

- 27 (32.5%) thought they spent too mujh time. 

0 thought they spent too little tiiP° 
6 (7.2%) did not respond. 

The payoff for the amount of effort was considered worthwhile 
Question 11 asked respondencs to indicate their perceptions of 
the payoff (on a scale of l=not worthwhile to' 5=extremely vorth- 

- 3.91 (very worthwhile) was the average value for the 
payoff for the effort of collecting the data. 
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since it is possible to produce customized reports from the 
Annual Statistics ^ respondents were asked in question 12 whether 
they regularly (1 to 2 times a year) requested such reports. 

- Only 3 (3.6%) respondents requested reports regularly. 

- 78 (94.0%) did not. 

- 2 ( 2.4%) respondents did not roply. 

Those who did not request reports were asked to indicate the 
reasons why. Table 5 - Reasons for not Requesting Reports 
summarizes the responses. 



TABLE 5 

REASONS FOR NOT REQUESTING REPORTS 



Reasrn for not Requesting 


Number 


% of resp. 






(N=78) 


Can get data myself 


34 


43.6 


Have no need for additional data 


27 


34 > 


Did not know I could do this 


24 


30.8 


Not sure what to request 


19 


24.4 


Data I need not available 


4 


5.1 


Too expensive 


3 


3.8 


Average number of reasons checked: 


1.3 





These data suggest that it may be possible to increase use 
of the customized report service. While the most frequent 
reasons for not requesting reports were that the respondent could 
get the data his or herself, or that there was no need for addi- 
tional data, the next most frequent reasons, that the respondent 
was unaware of this service or unsure of whal". to request, sugges^ 
that increased marketing of the service might increase its use. 
Further, respondents may not be aware of the types of data that 
could be supplied and how these data might be applied as a means 
for improving the decision making process within their library. 

Question 16 requested respondents to indicate their frequen- 
cy of use of the Annual St atistics . Table 6 - Frequency of Use 
°f the Annual Statistics summarizes the responses. The majority 
of respondents used the volume quite frequently, at least one a 
month (42.7%). 
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TABLE 6 

PRECUENCY OP CSE OP THE AITODAL STATISTICS 



Frequency Number 


of resDondents 


% of 




(N=82) 




once a month 


35 


42.7 


once a quarter 


22 


26.8 


once a Wv- ik 


14 


17.1 


once every 6 months 


9 


11.0 


once a year 


1 


1.2 


never 


1 


1.2 



In order to investigate whether those respondents who used 
the Annual Statistics once a week differed in the way they used 
the volume from other respondents, Chi Square tests ox indepen- 
de.ice were performed. Respondents were divided into two qroups 
those who had responded that they used the volume once a week and 
those who had checked any other response. This variable was 
cross tabulated with the uses for the volume indicated in ques- 
tion 6. No significant differences were founj between the types 
of use by those who consulted the Annual Statiscics once a w^ek 
and those who consulted the volume at some other frequency. 

This interesting finding suggests a possible lack of under- 
standing on the part of some respondents about ways in which the 
Annual Statistics could be used to facilitate decision making on 
a regular basis. Further investigation would be necessary to 
verify this supposition and assess whether additional education 
about the uses of statistics in decision making might change 
usage patterns. ^ ^ y 

Question 17 asked respondents to indicate their preferred 
month of receipt of the Annual Statistics . Table 7 - Preferred 
Month of Receipt of the An nual Statistics summarizes the respon- 
ses. Most respondents (40, 48.2%) preferred December or JanSary. 
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TABLE 7 

PREFERRED MOKTH OF RECEIPT OF THE ANNUAL STATISTICS 



Month 



January 

December 

March 

February 

September 

April 

October 

August 

wune 

July 

November 
May 



Number of 
Respondents 
(N=70) 

27 

13 

6 

5 

5 

4 

4 

3 

1 

1 

1 

0 



% of 
Respondents 



38.6 
18.6 
8.6 
7.1 
7.1 
5.7 
5.7 
4.3 
1.4 
1.4 
1.4 
0 



THE CATEGORIZATION SCHEI-IE 

Respondents were also queried on the current categorization 
scheme used in the tables. Question 13 providud a Likert scale 
which was ranked from l=not useful to 5=extremely useful. Table 
8 - Usefulness of Library Groups presents the number and per- 
centage of respondents who circled each value. The averagt value 
was 2.71 (betv/een somewhat useful and moderately useful). 



TABLE 8 

USEFULNESS OF L? RARY GROUPS 



Response 



Not useful 
Somewhat useful 
Moderately useful 
Very useful 
Extremely useful 



Numbtr of 
respondents 
(N=82) 

21 
14 
21 
20 
6 



S: of 
respondents 



25.6 
17.1 
25.6 
24.4 
7.3 



The somewhat bimodal distribution of responses caused us to 
question whether respondents who found the grouping useful 
utilizad the Annual Statistics differently than those who found 
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it not useful. Using Chi square tests of independence, we were 
unable to find significant differences between uses made of the 
Annual Statistics by the respondents grouped according to their 
responses on the categorization scheme's usefulness • Further 
investigation is necessary to determine why some respondents find 
the categorization scheme useful and others not. 

SDMKT^IZATION OF TRENDS IN OPEN-ENDED COMMENTS 

Open-ended comments were transcribed from all 96 question- 
naires received before May 10, 1990 and are presented in Appendix 
B, Comments. The major trends represented in the comments were: 

- Most respondents seem satisfied with the Annual Statis- 
tics , 

- There is concern about the need for standardized 
definitions. 

- The current grouping scheme of libraries is problema- 
tic. 

- New types of information should be incorporated into 
the Annual Statistics . 

In general, the tenor of the comments were positive. People 
seem satisfied with the Annual Statistics ; they rely on them 
"warts and all," and are willing to collect data and participate 
in the project. Responses to question 8 indicate that the 
statistics provided are quite useful in making the library's case 
to the administration and in determining the library's position 
relative to other libraries. 

Several concerns were raised by the respondents. Chief 
among these was the current method of grouping the libraries. 15 
responr'cnts provided comments on the categorization scheme. 
These comments were generally not favorable. Some respondents 
suggested that the categories be retained but that the data be 
presented in one combined ranking. Some respondents suggested 
other variables which could be added to the grouping formula such 
as public vs. private. 

The need for more standardized definitions and formulae for 
collecting the data was mentioned by 10 respondents to question 
18. This concern surfaced throughout the questionnaire and is 
represented in open-ended responses to questions 14 and 15 as 
veil. 

10 respondents also indicated additional types of informa- 
tion which should be reported in the Annual Statistics . The 
comments of all 10 reflected a desire for information on new 
technologies in use in the library; number of micros, availa- 
bility of micro labs, CD-ROM and locally mounted database usage 
were among the types of information mentioned. 
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Finally, 5 respondents asked for the Annual Statistics data 
to be made available in machine readable format. 



IV. FURTHER I^SEARCH 

This survey into the usage of the AAHSLD Annual Statistics 
has provided useful answers and perspectives into how AAHSLD 
member libraries use quantitative data in general and the Annvial 
Statistics volume in particular. After the AAHSLD Statistics 
Committee members review survey results as given in this report, 
additional analyses of survey contents may become evident. 

The survey has also provided some insight into areas where 
additional research might prove fruitful. These areas are 

- data reliability and validity, 

- the categorization scheme, 

- performance measures and t^xeir use, and 

the educational needs of members in the area of the 
usus of data and performance measures for decision 
making. 



While the survey was able to assess members' feelings about 
the reliability and validity of data in the Annual Statistics > 
it was not able to determine how accurately the data are being 
collected, or whether they are valid. Answers to these questions 
can only be found through on site investigation. 

Open-ended comments suggest that members are not completely 
satisfied with the categorization scheme as it now stands. At 
issue here is how best to provide comparative groupings and what 
criteria to use as a basis for such groupings. Additional 
research is needed to determine what variables members consider 
important in distinguishing medical libraries and whether it is 
possible to reanalyze the existing database to generate different 
categories. Techniques such as focus groups or individual inter- 
views might provide the necessary information. 

The survey results (ques*:ions 2-5) indicated that respon- 
dents were confused as to the delinition of a performance mea- 
sure. It is not clear from the survey whether respondents are 
also unsure about the application of performance measures in 
decision making. Further research, using focus groups or in- 
dividual interviews, might provide additional information about 
members* use of performance measures. This research effort might 
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assess member libraries' needs for performance measures and 
determine 



what measures could be produced from existing data, 
what measures would require additional data collection, 
and 

the degree to which member libraries really use and 
woulu collect performance measure data. 



The survey's results suggest that the AAHLSD or the Medical 
Library Associr tion might wish to consider the development of a 
number of Continuing Education modules in the areas of using data 
for decision making, uses and applications of performance mea- 
sures, designing in-house Management Information Systems (MIS) 
for use in library decision making and planning, and other 
related topics. Greater attention to educating library managers 
about performance measures and how such techniques can be linked 
to management topics such as evaluation of services, planning, 
MIS, etc. may be necessary if library managers (including depart- 
ment heads) are to better manage scarce resources in light of 
increased demands. 
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APPENDIX A 



THE USE OF THE AAHSLD ANNUAL STATISTICS IN MEMBER 

LIBRARIES 



Instructions: This^questionnaire is designed to assess the current use of the 
Annual Jtatistics and to solicit information on how the Annual Statistics could 
be altered to improve your use of it. Please refer to the 11th edition (1987- 
1988) as you answer these questions. Please answer all questions and feel 
free to attach additional sheets to provide further information. 



1. Within your library, wha. uses are made of quantitative data to 
improve decision making? Check all that apply. 

providing supporting data for budget requests 

providing data for intra-library reporting 

providing tocls to calculate management formulas 

establishing benchmarks for productivity standards 

establishing benchmarks to track the library's development over 

time 

providing salary information 

providing data for the library resources allocation process 

providing early warning s gnals of trouble spots in your library's 

operation 

providing comparisions between your library and other libraries 

calculating performance measures (data which measure the 

effectiveness and quality of library activities and services) 

other (please specify) . 

2. Does your library use any performance measures (data which measure 
the effectiveness and quality of library activities and services) to aid 
in decision making? Yes No 

If yes, what measures do you use? 



3. Should more performance measure data be included in the Annual 
Statistics! Yes No 

4. If yes, in which areas? 

Clientele (possible examples) 

Collections (possible examples) 

Expenditures (possible examples) 

Facilities (possible examples) 

Services (possible examples) 

Staffing and Salaries (possible examples) 
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5, Would you be willing to report additional data if it would allow the 
presentation of more performance measures in the Annual Statistics, 

Yes LNo 

6, How important is the Annual Statistfcs in providing information for 
the following tasks? Rank each on the following scale: (I=essential, 
2=somewhat important, 3=not important), 

providing supporting data for budget requests 

providing data for intra-Iibrary reporting 

providing tools to calculate management formulas 

establishing benchmarks for productivity standards 

establishing benchmarks to track the library's development over 

time 

providing salary information 

providing data for the library resources allocation process 

pro\iding early warning signals of trouble spots in your library's 

operation 

providing comparisions between /our library and other libraries 

calculating performance measures (data which measure the 

effectiveness and quality of library ac'dvities and services) 

other (please specify) ^ 

7, Is the amount of d?.ta being collected in the Annual Statistics 
too much just right not enough ' 

8, What data in the Annual Statistics do on find especially useful? Why? 



9. What data in the Annual Statistics do you find not useful? Why? 
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What is your estimate of the amount of time (in hours) spent in 
preparing data to be included in the Annual Statistics! 



This amount of time is: too much about right too little 

To what degree h the payoff for this level of effort worthwhile for 
your library? Please circle the r^'^ponse which best applies. 

•12 3 4 5 

J ! I J L_ 

not lomewhat moderately very extremely 

worthwhile worthwhiic worthwhile worthwhile worthwhile 



Do you regularly (1 to 2 times/year) request customized reports 

generated from the Annual Statistics! Yes No 

If no, check the reasons why. 

have no need for additional data 

too expensive 

not sure what to request 

can get the data myself 

data I need is not available 

did not know I could dc this 



Do you fi the groupings of the libraries into four categories (pages 
95-148) us< A for purposes of comparison? Please circle the response 
which best applies. 

12 3 '4 5 

J ! \ I L_ 

not lomewhat moderately very extremely 

useful useful useful useful useful 



In your opinion, how corisistenvly, from library to library, are data in 
the Annual Statistics being reported by the member libraries? For each 
category, please circle the response that best applies. 

Not Moderately Extremely 
Accurate Accurate Accurate 



Clientele 


1 


2 


3 


4 


5 


Collections 


1 


2 


3 


4 


5 


Expenditures 


1 


2 


3 


4 


5 


Facilities 


1 


2 


3 


4 


5 


Services 


I 


2 


3 


4 


5 


Staffing and Salaries 


1 


2 


3 


4 


5 
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15. In your opinion, how well do the data reallv describe the aspects of 
library activities 'hat they are intended to describe? Please circle the 
response that best applies. 

Not Moderately Extremely 

Accurate Accurate ' Accurate 



Clientele 


1 


2 


3 


4 


5 


Collections 


1 


2 


3 


4 


5 


Expenditures 


1 


2 


3 


4 


5 


Facilities 


1 


2 


3 


4 


5 


Services 


1 


2 


3 


4 


5 


Staffing and Salaries 


1 


2 


3 


4 


5 



16. How often do you refer to the Annual Satisticsl 

once a week once every 6 months 

once a month once a year 

Once every quarter never 

17. What month would te the best time of the year for your library to 
receive the Annual Statisticsl ^(month) 

18. How could the Annual Statistics be altered to improve your use of it? 
(e.g. format, types of statistics included, and instructional and 
explanatory material) 



Thank you for your time. We are interested in knowing any other comments 
you may have about the use and potential applications of the Annual 
Statistics. Please return the survey in the attached envelope by April 15 to 
Charles R. McClure and Carol A. Hert, Syracuse University School of 
Information Studies, Syracuse, NY 13244-1120. Thank you. 



APPENDIX B 
COMMENTS 



Comments ar'" recorded verbatim from questionnaires. The number of 
respondents was 96. The.'e comments include comments from 
questionnaires returned too late to be included in the quantitative 
analysis. 

QUESTION 1 - USES OP QC/JITITATIVE DATA - OTHER RESPONSES 

Inter-library and state agency reporting, establish cost 
recovery goals 

Providing easily accessible data to answer other surveys 

Address and phone numbers 

Beckground material on vacant positions 

Anticipate using for productivity and performance review but 
have not yet 

We are just beginning to work on these 

Accreditation reviews 

QUESTION 2 - DOES YOUR LIBRARY USE PEPJORMANCE MEASURES/ IP SO, 
imiCH? 

Haven't yet but probably will 

Collection development funds/ user; oparation costs/u^er; 
loans/user; searcbesi/user; I'm still trying to determine tne 
value and comparison from year to year of these figures 

111 statistics and lending/borrowing ratios with local and 
regional libraries 

Number of periodical subscrip'^ions 

Use of end-user systems (BAGS MEDLINE and BAGS Gurrent 
Gontents) by department and overtime. An increase in use 
equals a measure of ef f-^ctiveness 

RML reports, SLA documents, internal surveys 

Questionnaires, data from automated system gives usage for 
measuring effectively of services and collections 

Time/motion studies are done 
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QUESTION 2 (CON»T) 



No except superficially-because baseline data is so difficult 
to collect and comparative data even with Annual Statistics 
is so difficult to calculate 

In-house user questionnaire 

Use in ad hoc way, e.g. when I wcnder whether my ILL staff is 
sufficiently productive, I compare staffing information and 
activity information. I do not routinely monitor a predefined 
set of performance measures/ratios other than activity figures 

Journal use statistics, circ statistics, end user studies, 
user group studies 

monograph/total volumes; serials $/tctal $; ILL requests/ILL 
filled 

Serials expenditures to total acquisition expenditures 

We do not look closely at these* As yet I have not formally 
incorporated the data into planning, but plan to do so soon. 
I think these are some of the best statistics available. 

Monthly services statistics generated by each department, 
statewide automated ILS generates data 

Decline statistics for ILL; turnaround time, reasons for not 
filling, routing effectiveness, etc. 

Key area for us as I suspect for most people, it is one thing 
to count number of volumes on the shelves, it is quite another 
to be sure that one has the right volumes for one's clientele, 
but at least one can count volumes with some accuracy (or ILL 
tr=insactions, gate count, number of online searches) when we 
get to area of reference transactions we can't even get 
accurate counts. We gather rough estimates, and they are 
useful in helping us gauge how workload ebbs and flows during 
the course of the academic year, for example, but they don't 
really tell us how well we're serving our clientele. We wou±d 
be happy to expend more energy gathering data if we felt that 
the data could help us answer that question. 

Number of persons per area quantity produced in area (sh avers 
vs. number of items shelved) 

Performance Pleasures currently used for comparison purposes 
not decision making 
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QUESTION 2 (CON'T) 

User surveys, interviews, inventories, integrated system 
reports 

Reference transactions/reference desk hours; total public 
service activities/staff 

Circulation statistics for retention of serial titles, 
reference questions answered to determine staffing of 
information desk 

Serial expenditures, staff development and travel, main 
reference 

Basic unit cost studies 

Have looked at staffing and workload in ILL, database 
searching, reference service staffing and workload 

Results of user surveys, evaluations of instmictional 
activities 

ILL activity compared to ILL staffing 
Productivity to determine additional staff needed 
Medical schools in Canada; university libraries in Canada 
Surveys to clientele, self-studies 

We use productivity measures, the current raw tables provide 
data for us to compute for most performance issues 

I have trouble with your definition of performance measures. 
Do we have any that measure quality? 

Cost of serials by number of subscriptions 

Staff development, ILL requests and staff, generated income 

ILL turnaround time; negative feedback 

Ratio of print monographs to total print volumes 

Not derived from these data 

Some might be used if more information about the selection of 
respondent institutions was given 

User feedback and focus groups; fill rate; turnaround rime; 
surveys 
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QUESTION 3 - SHOULD PERFORMANCE DATA BE INCLUDED? 



If some standard measures were identified and applied 
consistently. ILL requests to total received is inaccurate 
because inconsistent and inaccurate reporting. 

Useful but situations differ so widely, it's hard t'o use for 
more than simply raising questions 

Probably not, but would like information on use of cd-rom and 
local databases 

QUESTION 4 - EXAMPLES OF POSSIBLE PEPJORMANCE MEASURES 

GENERAL COMMENTS 

The statistics should include detailed but simple instructions 
on sampling methodology which every library can easily adopt 
and religiously follow to assure quality data 

Percente.ge of students, faculty etc. attending classes 

Frankly I didn't look closely at the performance measures 
until leceiving this survey, some of them, such as monograph 
titles to total print volumes and database expenditures to 
accesses seem silly as reflected in the results. Clearly other 
factors seem to be at work. Some measures are interesting such 
as the amount of budget spent on serials and the number of ILL 
requests receiving to total sta:?f . 

Need some quantitative and qualitative norms to use as 
benchmarks. It will not be easy but it can be done 

Relationships between services and cost would be extremely 
helpful but not sure it*s possible to get meaningful data from 
a questionnaire with the variety of types of service 
organization, self service systems, remote access, eii.. Some 
attempt to assess amount of remote access might be worthwhile. 

EXAMPLES OF PERFORMANCE MEASURES RELATING TO CLIENTELE 

Ratio actual users to potential users 

# lib visits/# service transactions (all service transactions 
e.g. circulation, info desk, etc. ) ; circulation by user 
group/total user group 

We need help to increase our human resource allocation would 
like statistics that show size of professional and support 
staff compared to # of users (ex. 1 reference librarian per 
500 users o I think we all need such data.) 
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QUESTION 4 <CON»T) 

Breakdown of types of clients 
Users V. eligible clientele 
OPACS/clients 

Primary clientele/professional librarian 

Use relative to type of user 

Affiliated vs. others 

Potenticil vs. actual users 

EXAMPLES OF PERFORMAfTCE MEASURES RELATING TO COLLECTIONS 

$ spent per serial/number of current subscrip Ions by subject; 
$ spent per monograph title added/# titles by subject 

Serials/user; books/user 

size and outside use 

Collection expenditures to ILL and circulation activity 
Total volumes to total collection use 
Number book collect circ fm 1985 

Commitment of institution to computer based projects, cd-rom, 
interactive videodisc 

Access of online book file; nonbook files available though 
local online system 

Availabil ity v . rc^trieval ; degree of sub j ect coverage ; 
responsiveness to requests to purchase 

Serial subscriptions/primary clientele; monographs 
added/primary clientele 

Usage data 

Relation between collection size and use 

Usage study reports 

Current (last 10 years) size 
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QUESTION 4 (CON'T) 

EXAMPLES OF PERFORMANCE MEASURES RELATING TO LXPENDITURES 

Generated income expenditures to institutional expenditures; 
gifts and endowments expenditures to institutional 
expenditures; generated income expenditures and gifts and 
endowments expenditures to total expenditures; $ per paid 
subsciiption (average subscription cost) ; $ per paid monograph 
title (average monograph cost) 

Expenditure/user 

Ratio of salairy - collections - operating expenses 
Library exp' ^diture v'o total institutional expenditure 
Unit costs for various servicers 

Serials expenditure/serials and monograph expenditure 

Access V. acquiring expenditures 

Tech process staff cost^'ti :le, volume 

Expenditure breakdowns, etc. 

U.S. vs. foreign publications 

EXAMPLES OF PERFORMANCE MEASURES RELATING TO FACILITIES 

Study seats to primary clientele; sq. feet to primary 
clientele; micros to # of students; AV equipment to AV 
materials by format 

Percent of capacity used at peak periods/low periods 

Micro and AV facilities/students 

Space of library v total institutional space 

Ratio of seating, terminals, workstations to potential users 

Seating or sq. footage/primary clientele 

Study space v. collections 

Health science users to total seats or total ASF 
Microcomputer lab use 
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QUESTION 4 (CON'T) 

EXAMPLES OF PERFORMANCE MEASURES RELATING TO EXPENDITURES 

Generated income expenditures to institutional expenditures; 
gifts and endowments expenditures to institutional 
expenditures ; generated income expenditures and gi f ts and 
endowments expenditures to total expenditures; $ per paid 
subscription (average subscription cost) ; $ per paid monograph 
title (average monograph cost) 

Expenditure/user 

Ratio of salary - collections - operating expenses 
Library expenditure to total institutional expenditure 
Unit costs for various services 

Serials expenditure/serials and monograph expenditure 

Access V. acquiring expenditures 

Tech process staff cost/title, volume 

Expenditure breakdowns, etc* 

U.S. vs. foreign publications 

EXAMPLES OF PERFORMANCE MEASURES RELATING TO FACILITIES 

Study seats to primary clientele; sq. feet to primary 
clientele; micros to # of students; AV equipment to AV 
materials by format 

Percent of capacity used at peak periods/low periods 

Micro and AV £acilities/students 

Space of library v total institutional space 

Ratio of seating, terminals, workstations to potential users 

Seating or sq. footage/primary clientele 

Study space v. collections 

Health science users to total seats or total ASF 
Microcomputer lab use 
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QUESTION 4 (CON'T) 



EXAMPLES OF PERFORMANCE MEASURES RELATING TO SERVICES 

Typical levels of usage by different types of users 

ILL fill rate; ILL lequest per ILL FTE; reference transactions 
to hours desk staffp-d; searches per reference FTE; instruction 
contact hours per instructional FTE; total contact hours to 
total attending 

Quid pro quo sharing arrangements 
Computer searches 

Service statistics/users/hours/staff 

Automated functions v. service (e.g. online catalog to 
circulation) 

New services v. type of staff giving the service; trends in 
use of ILL to make up for lack of collection 

Use and effectiveness of reference 

ILL fill rates 

Collection use/primary clientele 
Education activities 
Types of 

Hours open to total collection use 

Reference, online searches 

EXAMPLES OF PERFORMANCE MEASURES RELATING TO STAFFING 

Staffing vis a vis ILL services, cataloging output, serials 
titles 

Total FTE to primary clientele; professional FTE to primary 
clientele; paraprofessional FTE to primary clientele; support 
FTE to primary clientele; circulation FTE to service hours; 
circulation FTE to total circulation and on-site use; staff 
development $ to professional and paraprofessional FTE, staff 
development $ to total recurring expenditures (exclude grants 
and contracts) ; hours information services available to total 
service hours; total FTE by program; cataloging FTE by titles 
cataloged 
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QUESTION 4 (CON'T) 

Staff tenninals/# staff 

Percent of professional staft to other staff 

Positions salaries tied to experience and level of job 

Relationship between # of reference librarians and hours of 
desk coverage 

Professional staff/total staff 

Professional/non professional; appropriateness of an extensive 
reserve service in a problem based self directed learning 
environment 

Percentage of total budget 

QUESTION 5 - WOULD YOU BE WILLING TO REPORT ADDITION DATA FOR 
PERFORMANCE MEASURE CALCULATION? 

Yes, if special measures do not have to be taken to collect 

Yes, if given a year's notice 

Yes, inform year in advance so statistics can be collected 

QUESTION 7 - AMOUNT OF DA'^A IN THE ANNUAL STATISTICS 

More data would be ni^;e if accurately reported 

Answer depends on what information, perhaps too much bare 
quantitative data, not enough qualitative information. 

The amount seems ok.. Whether we are collecting the right data 
is the big question. 

QUESTION 8 - WHAT DATA ARE ESPECIALLY USEFUL? 

E!:penditure means in justifying budget r^quescs. categories 
of libraries, however group different institutions and imply 
they are all comparable, so I usually calculate my own group 
means 

Personnel, facilities, some service measures all for 
comparison purposes; a few of the performar.ce mc^asures now 
reported both for comparison purposes as well as for setting 
performance standards 

Expenditures for collection development and salaries 
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QUESTION 8 (CON'T) 



Dollar allocations: e.g. staff development, acquisitions, 
salaries for budget comparisons, information for a 5 year plan 
etc., also size (# of volumes and physical size) to compare 
for possible new building 

Comparative on journal holdings, expenditures for data base 
searching and ILL (responsiveness and collection integrity) 

Number of library materials added annually, book and 
periodical budgets, salary information 

Most of the data are scanned carefully for comparison 

Collection, reference use, expenditures data 

Salary data; colloction size data 

Collection development data; personnel allocations, salary 
information 

Collection size rankings, salary and budget information, also 
ILL fi?l rate rankings; these have been useful in preparing 
operating budgets, justifying budgets and space needs 

Salary, # of staff, ranking of colleges in all areas of the 
statistics collected 

Comparisons with other libraries in same state-competition 
over monies 

Supporting data for budget requests and salary information is 
very helpful, unfortunately while administration listens and 
seems supportive they don*t give more funds because they say 
they don't have more funding 

Budget, salary information to provide clout for financial 
support 

Expenditures, staffing, service data, institutional data 

Acquisitions budgets, I base much of my budget request on this 
information. As \xiy library reoides within main university 
3 ibrary much of the other data cr.n not be compared 

# of current serials - most important statistics (the one that 
matters to dean of medical school) ; # ILLs - good measure of 
work and collection weakness 
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QUESTION 8 (CON'T) 



Salary information was significant in recent request for 
librarians ' salary market adjustment. # seri.als received 
compared to expenditures for serials for comparable libraries 

Salary and budget for budget justification 

Budget and salary information, comparisons between my library 
and others are useful as supportive data for budget requests 

Salary figures-remaining competitive in recruiting; 
collection-figures how well are we doing vis-a-vis our peer 
group; others are more for yo''r information but we want to 
know about othex* libraries 

Apparently hard dat collection size, space, visits, and 
expenditures. While ley may not be as hard as they appear, 
my administration understands and responds to these figures 

Space (do branches have special functions and purposes) 
staffing (number FTE) and expenditures, collections data, and 
services activities 

Salaries, budget, staffing for comparisons for manager.»ent and 
budget preparation 

Acquisitions , personnel 

Ranking tables, provide benchmarks and trends; budget figures 
also 

Service, collection, staffing, all for budget preparation 
Expenditures, staffing, ILL, salaries, collection 
Budgeting, staffing, salaries 

Ability to compare across sppctrum of library processes, 
services, acquisitions, space, etc. with libraries supporting 
similar programs 

One is constantly challenged by dean and directors needing 
some statistic in 24 hcurs, any ongoing source of statistics 
is useful 

Collection development statistics (count) and expenditures; 
salciry tables, these are used for bench mark comparisons 
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QUESTION 8 (CON'T) 



Rank order tables are extremely useful for comparisons; The 
Annual Statistics are recognized as valid comparative data 
for supporting budget requests, salary information. Most of 
the tables I use to provide information for other surveys-do 
not have to reinvent the wheel; Every ye.ir a different piece 
of information from Annual Statistics is used to support, 
defend, or promote a service of activity 

Many of the rank order tables help justify budget or staff 
requests let us know in relationship to other libraries 

Budgets especially acquisitions - use in budget request 
presentation; facilities - use data for new library building 
request; staffing size v. services provided shows 
administration where we stand, how we perform compared to 
other libraries 

Expenditures-binding I have never needed to justify or request 
additional funds for binding 

Salary, cq,, fee^. staff size, budget breakdown all used to 
convey our problems to administration 

Monies spent on collections 

Salary survey supports topic for which I have needed 
information. Other tables could be useful, just haven't needed 
them for support. As a new library director more of this data 
will be meaningful to me. 

Ranking tables grouping with other libraries 

Expenditure data for use in budget requests, salary data to 
help in justifying salary adjustments, usage data to show off 
activity levels to administration, personnel data (this is 
useful whe*. compared with activity levels, such as ILL 
performance measure 

Staffing levels and productivity tablas 
Salaries, budget, staffing 
Financial data 
Rankings 

Expenditures and volume of operations and services 

Salary, staffing, services: comparison with like institutions 
help justify our budget requests 
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QUESTION 8 (CON'T) 



Salaries, accraisitions budgets, personnel breakdowns 
We never use it 

Salary comparative data for like libraries, expenditures, 
services data 

Staffing levels, ' service and other output counts 
Ranking within groups 

I've used almost all of it at one time at another. Its all 
been useful 

Most all of it has been useful at various times including 
expenditures, clientele served, collection, staff ing. 
Information on how heavily services are used (circulation, 
reference, etc.) is less useful all by itself. 

Service/usage gate count, circulation, online services, etc. 
allocations for workload comparisons when also looking at 
staffing statistics 

Salary data to argue for more $$; client population 
comparative data to argue for increased funding for health 
science libraries based on peer libraries 

Expenditures for budget comparisons; salary information; 
circulation and collection statistics for comparisons 

Financial information to strengthen our case for greater 
funding; salary information helps us in our riving for 
improvement; collections-helps us in our development 

Expenditure information, acquisitions statistics, service 
statistics, some performance measures 

Salaries and expenditures - for budget justifications; 
Services - for comparative analyses and to identify libraries 
with similar service levels to consult for specific issues as 
needed 

Expenditures/ category ; ILL perfonnanc(S measures; Service 
statistics. Useful for budget preparation and staff 
allocation. 

Comparison and ranking tables. Salary data 
Salary, holding, services data 
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QUESTION 8 (CON'T) 



I've used almost all of it in the past two years of doing 
building plans, budget analyses, resource allocation, reality 
checking 

Salary data. Please, please encourage directors to share 
their own salary data. I just got a two year 33% salary 
package for my professional librarians based in part on the 
Annual Statistics . My own raise was limited to the standard 
6% administrative increment in part because I could not make 
a good case for ineauit^ because other directors don t report. 

Primary clients , irionograph titles , serial titles , 
expenditures , salaries and wages . This data is used as a 
basis of comparison in our budget request each year. 

Salary tables. Budget allocations 

Salary data for comparison purposes; collection budget and 
salary budget for comparing our budget 

Total bulget, acquisitions budget, staff size, and service 
statistics are useful for providing comparative data for 
budget requests 

Budgets - use the figures to derive "norms"; staffing - but 
I would still like breakdown into libraries that carry tech 
seirvices and those that do not; current journals - as "norms" 

QUESTION 9 - WHAT DATA ARE NOT USEFUL? 

I have no idea how other people are report ing the data ; 
categories of libraries, gross volume*; v. titles-these 
definitions have changed over time and ^^o not allow for 
retrospective comparisons; not sure everyone is using the same 
definition now; no record of in-house database searches 
(MEDLINE subsets, CD-ROM etc.) 

Most collection data since they are highly site specific 

Least used information is on c irL.alation, photocopies, and 
library uses 

Breakdown of reference transactions, photocopy exposur^^s, gate 
count 

Inrcrmational and directional numbers, these have no apparent 
usefulness and are highly affected by location, image, ^.tc. 
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QUESTION 9 (CON'T) 

Groupings into categories-' doesn't fit 

Some of the data requested is obtained by using ron standard 
internal statistics and I question the reliability and 
validity of data from libraries I know to function 
differently than is reported. 

Sq, feet of space, clientele 

Big is not always better 

Management formulas are not helpful becau^se we are barely 
getting by and this kind of information doesn't seem 
persuasive in improving our situation 

Productivity standards bench*aarks data for "Library resource 
allocatiop not useful to this library because the staff 
conscientious and budget is not large enough to need to worry 
about allocation of resources 

I have no way of singling out such statistics as circulation, 
ILL, photocopies, patron count, etc. 

# current serials -institution only interested in current 
journal subscriptions, gate count, collection use, photocopy 
exposures (not very scientific, best we can do) 

Missing data which means comparisons cannot really be made 
(e.g. one school may not be charged for computer center cost'^ 
while another is) 

In house use, I suspect these numbers are fraudulent 

Detailed breakdown of allocation of staff time, data is 
dependent on services and operations in each library, these 
vary greatly 

Reference activities: suspected lack of uniformity in 
reporting and inadequate operational definition; number of 
serials titles data: suspected lack of uniformity in 
reporting, no definitions 

Breakdown of stafj^ by function is tedious to do and I"ve never 
used the info although T do compare levels of service with 
numbers of staff. Too variable; rank order tables were more 
helpful when all libraries ranked uncategorized; expenditures- 
building costs are questionable; what is included varies so 
form institution to institution, omit 
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QUESTION 9 (CON'T) 

Grants and contracts personnel (table 41) , combine tables 
37,38 

Database search counts- too many factors influence #; 
photocopies-no real purpose; operating expenses-hard to 
capture and may not necessarily impact directly on each 
library's budget 

Information on some services, such as database accesses- a 
poor searcher may access databases unnecessarily-not a good 
marker for quality or quantity of searches 

Clientele, withdrawals, monograph titles, s^. feet, seating 

For small library (1 person) the breakdown of FTE support for 
specific functions not that valuable 

Performance measure tables-have never made or needed to use 
this type of data 

Volumes in off site storage-not relevant to our needs; 
photocopies-means nothing, not tied to anything (e.g. # of 
machines, revenue, use of off site photocopies, etc.) 

Some performance measures because inconsistent reporting or 
not taking into account programs served, restrictive policies 
etc. 

Reference transactions and directions; transactions are 
counted or estimated in different ways by different libraries; 
there is enough variation in these numbers to make comparisons 
virtually impossible. 

Free standing private institution. Some data doesn't pertain 
to a library, some data provides targets/goals for us but are 
not attainable now (e.g. budgets) 

Breakdown of how many FTE's in various professional categories 

Statistics that we do not keep, that we are obligated to 
report; also question reliability of data. Title counts not 
useful 

All data collected now useful 

Number of database accesses, irrelevant to activity actually 
taking place 



37 



41 



QUESTION 9 (CON'T) 



Categorization of libraries; we compare to other libraries so 
we recalculate rankings 

Year library was established; available elsewhere 
Staffing plans 
(illegible) days 

Item level counts (e.g. # microforms); I*m not convinced that 
uniform interpretation/reporting is occurring in these counts 

Clientele cannot be a correct figure; this statistic should 
be an estimated percentage of use by certain clientele instead 
of vaw; gate count is superficial; reference transactions 
without a uniform formula; this figure is nebulous 

Tables in Annual Statistics should also report medians. The 
distributions are highly skewed and they could be more useful 
than means in explaining distributions. 

Archives and manuscripts because we do not have a collection 
in this area. 

Probably the high variability in clients reported plus the 
difficulty in dealing with large numbers of unreported clients 
for libraries on general campuses makes this number hard to 
use. 

Breakout of FTE, we are tod small, distribution of staff in 
categories difficult 

Don't use all the data, but not willing to say that any is 
not useful 

Number of staff by function, not tied to quantitative data of 
nuiiber of service units provide'", or materials processed. 

Gate count - too inaccurate; total primary clients - o 
inaccurate due to large variation ^ in hospital staff counts; 
directional transactions and total information contacts - 
directional transactions counts are not reliable data; 
database accesses - no longer important with implementation 
of end-user search systems 

Salaries - little use as we operate under a faculty 
association contract 
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QUESTION 9 (CON'T) 



Rank order tables according to library ranking groups. We 
prefer one overall ranking. The information about personnel 
activities, such as Table 36, is entirely too detailed. 

Performance measures tables - too many variables affecting 
reported statistics to make statistics meaningful 

Grants and contracts perrionnel (has little im^^^.ct on overall 
staffing profiles), no. of photocopies (who cares?) 

QUESTION 10 - ESTIMATE OF AMOUNT OF TIME SPENT PREPARING 

Including recording of statistics, its enormous, SAMPLING! 

The large amount of time is due to our being integrated into 
the main library 

Completing surveys doesn ' t include collecting statistics 
throughout the year 

Pulling together info and filling out forms takes about 20-25 
hours. This does not seem like an undue amount of time. Of 
course, that does not include all of the time that is spent 
throughout the year maintaining the records that give us those 
results . 

QUESTION 11 - THE DEGREE TO WHICH THE PAYOFF IS WORTHWHILE 

Worthwhile enough, so we do it, but we cannot increase the 
effort. We must switch to sampling so less effort can yield 
more data. 

QUESTION 12 - DO YOU REQUEST CUSTOMIZED RPORTS GENERATED FROM THE 
ANNUAL STATISTICS? 

Would like machine readable summary data 

Also need data in MAC format rather than IBM 

Not enough indepth evaluation occurring to require additional 
data. We hope to change this over the next year. 

So far, I've done this n.yself but I believe these reports can 
be useful 

QTJESTION 13 - DO YOU FIND THE GROUPINGS INTO CATEGORIES USEFUL 
Machine readable data would allow customized grouping 
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QUESTION 13 (CON'T) 



Do not like this, does not help explain rankings to dean 

Not useful and completely irritating 

Do not like this arrangement-like one list 

Mean anu medicin information for entire data set also useful, 
would also like to see public w private, residential v. non 
residential 

Very useful when Canadians were included 

QUESTION 14 - liRE THE DATA BEING REPORTED CONSISTENTLY? 

It is difficult for any one library to say how accurate 
others ' reports are . Editors can answar this better than 
others. These onswers are from an editor. 

I am of the optimistic belief that each library provides the 
best possible data they can. I uo not feel qualified to judge 
accuracy. I take the information at faca value with allowances 
for imperfection. 

I would have to assume it is reported accurately. We have 
found errors in our own statistics from time to time. 

I can not answer this. I have to ass-ame each person answers 
to the best of her ability. 

No way to tell. I suspect that the margin of error is fairly 
large. 

Having edited questionnaires for 5 years I have found some 
gross errors and serious lack of inteiest in collecting them 
from a subset of the libraries (5-10-%) they submit #s but 
don't seem to care or perhaps know how to riake them r-asonably 
accurate. Having filled out questionnaire 12+ times myself, 
however I know that some of the request e^i numbers are 
difficult to determine accurately. 

What is a serial , FTE are the problem. I was surprised to 
learn that Dick' s library counts all directional questions, 
but I nave never considered counting any but those put to 
reference desk staff when on duty. 

I feel the high level of reliability of AAHSLD data ir due to 
careful editing of each response. Some arear; are soft because 
either institutional data ar<=» bad or hard to come by or we 
haven't given reporters good ways to measure consistently. 
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QUESTION 14 (CON'T) 



Salaries, accurate; staffing counts by function are very 
subjective. This is an internal problem rather than one for 
the survey; we still do not have some of the data from the 
University library's accounting system. 

QUESTION 15 - HOW WELL DO THE DATA DESCRIBE WHAT THEY ARE INTENDED 
TO DESCRIBE? 

I suspect that most of the inaccuracy on the part on 
individual librari<=^.s tends to even out 

Inter-institutional variability appears high. Incra- 
institutional appears low, so trend analysis should be okay. 



QUESTION 18 - HOW COULD THE ANNNUAL STATISTICS BE IMPROVED? 

Improve definitions of data to be reported, Droi:> the 
categories or restructure them to consider more variables that 
distinguish institutions - private v public, enrollments, etc. 

Reduce data on collections and combine microforms with print 
(eliminate #55-61, 69, 71, 73-4); eliminate AV from 
expenditures? , include with print (92-95) ; eliminate binding 
include with other operating expenditures (104-107) ; eliminate 
contracts and grants; eliminate photocopies (218) ; eliminate 
orientations (234-235) keep one category "instructional 
programs" and let each library define it with some general 
guidelines. Many lib orientation programs have substantial 
"instructional" content 

Don't like four categories would rather see one listing for 
all, everything else okay 

Quite satisfied 

Its focus is good in the present format. I would not encourage 
more performance measure data but that type of information 
would be helpful in a separate publication. The real problem 
is getting libraries to report data accurately. 

I realize that changes occur, but it would be nice if for some 
years the data requested remaiiied constant. We have to fill 
out a number of questionnaires, and they all differ to some 
degree about the same data. This makes for extra work, and 
makes it difficult to collect data from yoar to year. 
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QUESTION 18 (CON'T) 



Go back to old ranking format. Have people report accurate 
data such as salaries-in this region I personally know that 
several directors do not report their true salaries which are 
composed of a base salary and supplementary pay. Currently 
they are reporting only their base pay and not the supplements 
they receive 

Complete coverage of all medical libraries 

Get rid of the groupings, they have little meaning in major 
medical schools 

The format is good. I like the ranking tv^bles, definitions 
are good. 

Standardized sampling instructions, machine readable report 
distribution 

Highlight name of rank order tables, e.g, rank order of total 
net assignable sq. feet; name tables in non-librarian 
(administration-friendly) terms (e.g. rank order of total 
facility size) Tab sections better and use larger size page, 
heavier paper for section dividers; invaluable essential data, 
warts and all 

Give us the data in machine readable form. I need a list of 
each question number and the name associated with the number, 
instructions (general for each number) , library number and 
name, number for each library/question. I will write a 
program to enter data and produce my own performance measures 

Remove osteopathic schools and all libraries that support 
anything less than a complete 4 year medical program. (such 
as UK Wichita) . As an alternative isolate them and report 
separately. 

A breakdown by RML region would be useful; include more 
operational definitions; would like to see also total serial 
titles currently received, including duplicates, gifts, etc. 

Omit categories, or include straight ranking; Include data on 
provision of technology to users, micros, media labs, # PC's 
CD's, etc. 

Need to revise public services statistics, i.e. libraries 
using CD-ROM for end user free searching, impacts on database 
accesses and online search statistics and budget; clearer 
definitions or standardization of services (getting better 
now) 
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QUESTION 18 (CON'T) 



I locate what I need with little trouble. 

Salaries by area not consistent with anything. We (KY) are 
still listed in area 5 which has not been accurate since the 
NLM merger of regions • Our benchmarks are also in areas 
4,6,7,8. Either change the areas or give one national table 
like ARL does. This is the most frustrating part of the Annual 
Statistics . I have to try to utilize for our benefit of 
comparisons. Thanks for the opportunity to bring this up. 

A fine job already in presentation; salary data for all areas 
would be more useful if a separate composite for ranked 
respondents, total, mean, median, standard deviation could be 
added excluding Canada data . Canada data skews U.S. 
comparisons 

Biggest aid would be to gather information that measures 
effectiveness. Not sure AAHSLD stats are an appropriate tool 
for this however. I've not come across any descriptions of 
how to measure effectiveness routinely. In lieu of that, it 
rdght be worthwhile to downplay what we do keep in the way of 
patron interactions. Sampling actual patron interactions 
without attempting to define the minute differences between 
reference and directional questions would at least give us 
workload trends. I do use the Statistics and find them 
valuable. Look forward to seeing how they may be improved 

Prefer not to have categories which are based on criteria that 
do not apply to us (special reports could be done for this) 
Annual Statistics could change its format to regional, state, 
something more meaningful 

Committee obviously feels it is necessary to constantly tinker 
with the Annual Statistics not always in a positive way. Basic 
comparative statistics useful for most. If it is felt that 
more and more information should be added to an already 
saturated database then perhaps a separate publication should 
be considered. The statistics serve us well now. Kowev^*r, 
more than a questionnaire is needed if the scope and purpose 
of the statistics are to be entirely changed. Shouldn't this 
be discussed in depth at the AAHSLD meeting? 

A different type face that is more readable and darker print 

Make clear how things are to be counted. (Are some people 

counting, all others sampling for use of in house materials, 

same for reference statistics?) Maybe everyone should be 
sampling but more consistently. 
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QUESTION 18 (CON'T) 



Although I find the Statistics frustrating at times I believe 
they are beneficial to all of us. Certainly they present 
inaccuracies at tiinei> but they are better t an many other 
compilations. The month of receipt is not at» important and 
the FY. We have a reporting year and a FY thus we are always 
slightly off. 

Eliminate the ranking categories, list =^11 libraries in one 
list 

For us it would be most useful to be able to break out all 
private independent medical schools 

Greater attention to services provided: eg. microcomputer 
consulting, fee based reference; better information on local 
online resources eg. MEDLINE, MICROMEDEX, CINAHL, 

Do away with grouping of libraries. Have more accurate count 
of people using library (gate count doesn't give true picture) 
More information on names of ILS used. More statistics for end 
users and include special section on CD-ROM. Also type of 
system: mainframe, micro, mini. 

Well funded prestigious libraries are easily identified. Less 
well established or r^upported libraries could be recognized 
in some way for progress being made, contribi^tion to the 
profession, higher than <inticipated rankings, or other 
significant ways in which they contribute to the total medical 
library scene 

National ranking rather than group rankings or in addition to 
the groups 

Sometimes it is useful to compare within category but a.. so 
want absolute rankings 

Please add overall rankings regardless of grouping. 
Administratively the grouping is counterproductive when it 
comes to using the statistical information to make a case 
before medical school administration. Medical school 
administration rank themselves among other medical schools 
nationally and their ranking has no correlation to the 
grouping you use for medical libraries. Also are statistics 
in a package manageable enough to produce disks in LOTUS 
format, so that we could purchase the disks and manipulate 
the data our .elves? 
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Volume count is no longer an adequate measure of success of 
library. Wanw to see more ways in which services can be in- 
cluded: networking, lAIMS projects, links outside walls, etc. 

Format's fine but could use improved printing and typography 
(how about using a MAC) . Could use data on use of CD-ROM and 
local databases 

More productivity measures 

Micro version which is compatible with LOTUS or other 
spreadsheets, online or diskette submission of date: 

I was part of editorial board when we went to ranking 
libraries in groups. I think it is interesting to know what 
libraries are in what group (especially my own) however I 
think that the rank order tables should go back to a single 
table -perhaps with the group number indicated. I find that 
when I do comparisons I look across all groups for absolute 
ranking. Our institution tends to compare itself to certain 
others which is based on other criteria than those related 
strictly to libraries. A single table showing who is in what 
group would be helpful, separate from the ranked tables 

More information on outreach roles of libraries: e.g. 
involvement with faculty on faculty-wide but library related 
projects; educational activities undertaken, impact of 
educational endeavors 

We need to be notified before the questionnaire goes out if 
definitions will change. Our statistics are set up as we 
expect you will require them. Please notify early. Also 
annual budget information is not really accurate for libraries 
on a biennial budget 

Continuous changing of types of information wanted/in which 
form needed is a constant headache. Please stick to certain 
types of statistics for several years. If you need to change 
statistical information, how it is collected or add several 
questions please inform libraries at least one year in advance 

Better instructions on collections; institution identity and 
library characteristics (appendix A) ; drop medical school 
program (virtually all are full) , institution type (widely 
known or ascertainable elsewhere) ; add date of opening of 
current facility; date of significant expansion or renova- 
tion; insist that "year established" means year library was 
formed/chartered as entity regardless of physical location. 
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I Vrould like other information collected in conjunction with 
Annual Statistics ; e.g. number of computer work stations 
available to public, number of CD-ROM subscriptions and work 
stations, also a regular report on automation activities; 
specifics of systems not just yes or no on whether something 
is automated. Information on support staff salaries would be 
helpful 

I would like more info on use of audiovisuals; information on 
database use by endusers should be addev''; the statistics 
included tend to be very conservative; There should be more 
information about more recent trends in libraries as a 
separate section (ex. CD-ROM, consortium, outreach and 
marketing, in-house mainframe use of databases, microlabs, 
etc.) ; medians should be included in the general tables. The 
distributions are highly skewed. 

Establish a standard measure or measures for "unit? of work, " 
"units of information*" if possible. So much of what is 
reported could be normalized and we could report dollars spent 
to provide X units of information using X units of work, etc. 

Discontinue the library groupings. Our administration 
compares itself to other leading schools of medicine as 
benchmarks, evvsn if they are not comparable by the criteria 
used to group libraries. Also, the groupings are difficult 
to explain to administrators who have short attention spans. 

Fine, thanks all. 

Add a que-*tion on status of librarians (i.e. faculty or 
professional status) . 

See no need to alter. 

Find a way to include Canadian libraries in groupings; break 
out staffing (and associated costs) by whether or not 
technical services are included 

We wish each edition had a brief overview at the front which 
gave facts such as number of libraries responding, impact of 
changes such as adding the Osteopathic libraries, key 
averages, changes of these key numbers (total budget, 
acquisitons budget, collection growth) . This does not need 
to be long, but we think it would be useful. We also wish 
there would be information about the current year's bixdgets. 
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QUESTION 18 (CON•T^ 



I feel the tyjDes of statistics and the format are basically 
appropriate. A more timely publication an distribution 
timetable would enhance the value of t^xe statistics 
considerably. A projected collection (acquisitions) budget 
would be helpful. We are reporting current salaries and a 
projected current acquisitions budget would also be useful. 
A breakdown of that category into five areas (monographs , 
serials, MRDF, AV, and computer software) would, of course, 
be even more useful. 

It could contain a paragrah on each library reporting. The 
paragraph would contain facts on goverance, relationships to 
off-site or other campus libraries; and services and 
functio*.. ; including automation, provided from central library 
or university resources. Without the context, many of the 
rankings or numbers are confusing or misleading. 
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